1. Field of the Invention
This invention relates to a camera system which is composed of an image sensor unit and a camera DSP. This invention realizes a high-level and intelligent camera system with scene analysis and classification with keeping flexibility of system configuration for system requirement and future expandability.
2. Description of Related Art
The existing camera unit is composed of the image sensor such as CCD (charge coupled device) or CMOS imager, AFE (Analog Front-End), and camera DSP. In this camera DSP, the basic camera related functions such as Auto White Balance (AWB) for changing color according to the color environment for better color quality. Auto exposure (AE) for fitting to light exposure condition, and Auto Focus (AF) to adjust the focus to the on-focus position for a certain object are processed. In addition to these functions, several other functions are needed to be processed on camera DSP. The camera DSP is divided into two major parts, one is the dedicated hardware block for data path part in which there are filters and other hardware processor units. And also the dedicated hardware block has some kind of measuring unit which extracts some kind of parameters related to the feature of picture for the sake of camera process controlling. The other part on camera DSP is the processor part, which is realized as DSP (Digital Signal Processor) or on-chip CPU The DSP/CPU is usually used for sequence control, adaptive control and some judgment for camera processing control. The DSP/CPU gets parameters from the measuring unit and calculates and selects parameters which will be set into the parameter register of hardware block to control the process and to adjust the process for better picture quality.
FIG. 5 shows an example of a conventional typical camera system. The reference numeral 1 denotes an image sensor unit, in which CCD or CMOS image sensor is usually used. In FIG. 5, this image sensor unit also includes lens module with focus control function and iris for exposure control function. The signal 8 is the control signal for focus motor and iris actuator and son on. The analog output signal 2 from image sensor unit 1 is input to AFE 3 to convert an analog video signal 2 into a digital video signal 4. The digital signal 4 is the input to the front-end camera DSP 5, and is input to the camera signal data path 10. In the camera signal data path 10, some feature values are extracted with the measure unit 11. These feature values are sent to DSP 12 and are used for estimation of parameters to be set into the parameter register 13. The values in parameter register 13 are used to control camera signal data path's operation. The digital video signal 4 is processed at the camera signal data path 10 and the result is output as a video signal 6 and is sent to a next stage. For some application such as cellular phone, one of the common functions of the back-end functions is the image compression. In this case, the video signal 6 is the input of conventional processor 7 and is encoded to a compressed data signal 9.
One of the conventional camera system's typical characteristics is that the camera DSP is a stand-alone type. This means there is no strong cooperation with the back-end processor for camera processing. The front-end camera DSP is doing all of camera processing and will output the processed video signal. The back-end block will not cooperate with front-end camera DSP, and the back-end processor just uses the output from the front-end camera DSP. Only the initial setting of the register is done by an external host processor or a back-end processor. However, there is no feedback from the back-end processor to control the front-end camera DSP unit. This makes easy to use camera font-end DSP since it works independently for a stand-alone function. However, it is difficult to integrate powerful function into camera DSP because of limited DSP's performance and limited memory size of camera DSP.
Some of the processes on a camera system are strongly scene dependent. In AWB case, for instance, the best condition of AWB is completely different between in an outdoor and in an indoor situation. The camera system extracts some kind of parameters to judge if the scene taking is indoor or outdoor. Sometimes, however, the judgment becomes wrong because of an unexpected scene in developing phase of AWB algorithm in advance. These misunderstanding of situations make image degrading into strange color and irrelevant color.
To avoid these situations, camera DSP has been implemented so as to cover a diversity of situations with adding special conditions into judging algorithm. But as there are so many situations in real operation, it is impossible to cover all situations by just adding special conditions to the algorithm. And since there are also the limitations of memory on chip, DSP's performance limitation and so on, it is difficult to realize these complex algorithms with small camera DSP unit.
These situations are the same as those for other camera processes. In AF case, without any information about an object to be focused and its position in a picture, focusing on this object is very difficult. If the object is set in the center of a camera scene, it is relatively easy to focus on it. But if the position of the object is not in the center of a picture, AF system cannot detect the object to be focused and therefore, it is impossible to focus on it.
For AE case, the situation is almost the same. When the scene is in a backlight condition for example, it is hard to properly adjust the exposure to realize a good picture quality.
For the camera processing, basically very high performance of the DSP or CPU in the camera DSP unit is needed for judging the situations of scenes because the conventional system with limited DSP/CPU power cannot realize scene dependent processing properly.